gitlab.com/SiaPrime/SiaPrime@v1.4.1/doc/File Contract Negotiation.md (about)

     1  File Contract Negotiation
     2  =========================
     3  
     4  Securing data on SiaPrime requires creating and revising file contracts in an
     5  untrusted environment. Managing data on SiaPrime happens through several protocols:
     6  
     7  + Settings Request - the host sends the renter its settings.
     8  
     9  + Revision Request - the renter will send the host a file contract id, and the
    10    host will send the most recent file contract revision that it knows of for
    11    that file contract, with the signatures. A challenge and response is also
    12    performed to verify that the renter is able to create the signatures to
    13    modify the file contract revision.
    14  
    15  + File Contract Creation - no data is uploaded during the initial creation of a
    16    file contract, but funds are allocated so that the file contract can be
    17    iteratively revised in the future.
    18  
    19  + File Contract Revision - an existing file contract is revised so that data
    20    can be added to an arbitrary place, or removed from an arbitrary place.
    21  
    22  + File Contract Renewal - an existing file contract is renewed, meaning that a
    23    new file contract with a different id is created, but that has the same data.
    24    New funds are added to this file contract, and it can now be modified
    25    separately from the previous contract.
    26  
    27  + Data Request - data is requested from the host by hash.
    28  
    29  + (planned for later) Storage Proof Request - the renter requests that the host
    30    perform an out-of-band storage proof.
    31  
    32  + (planned for later) Metadata Request - the renter requests some metadata
    33    about the file contract from the host, namely the list of hashes that compose
    34    the file. This list of hashes is provided along with a cryptographic proof
    35    that the hashes are valid. The proof is only needed if only a subset of
    36    hashes are being sent.
    37  
    38  A frequently seen construction is 'acceptance'. The renter or host may have the
    39  opportunity to accept or reject a communication, which takes the form of a
    40  string. The acceptance string is always the same, and any string that is not
    41  the acceptance string is a rejection. The rejection string can include reasons
    42  why the rejection occurred, but most not exceed 255 bytes. After a rejection,
    43  the connection is always closed.
    44  
    45  The protocols described below are numbered. The number indicates when the
    46  communicator is switching. Each pair of numbers is a full round trip of
    47  communications.
    48  
    49  All communications attempt to support slow connections and Tor connections. Any
    50  connection with a throughput below 100kbps may struggle to perform the uploads
    51  and downloads, and any connection with a roundtrip latency greater than 2
    52  minutes may struggle to complete the protocols.
    53  
    54  Settings Request
    55  ----------------
    56  
    57  The host signs the settings request to prove that the connection has opened to
    58  the right party. Hosts announce on the blockchain and perform burn, therefore
    59  identity is important.
    60  
    61  1. The renter makes an RPC to the host, opening a connection. The connection
    62     deadline should be at least 120 seconds.
    63  
    64  2. The host sends the renter the most recent copy of its external settings,
    65     signed by the host public key. The connection is then closed.
    66  
    67  Revision Request
    68  ----------------
    69  
    70  The renter requests a recent revision from the host. Often, this request
    71  precedes modifications. A file contract can only be open for revision with one
    72  party at a time. To prevent DoS attacks, the party must authenticate here by
    73  performing a challenge-response protocol during the revision request. Putting
    74  this challenge-response requirement in the revision-request can help improve
    75  privacy, though the host is under no cryptographic or incentive-based
    76  obligation to preserve the privacy of the revision.
    77  
    78  1. The renter makes an RPC to the host, opening a connection. The connection
    79     deadline should be at least 120 seconds. The renter sends the file contract
    80     id for the revision being requested.
    81  
    82  2. The host writes 32 bytes of random data that the renter must sign for the
    83     renter key in the corresponding file contract.
    84  
    85  3. The renter returns the signed challenge.
    86  
    87  4. The host verifies the signature from the renter and then sends the renter
    88     the most recent file contract revision, along with the transaction
    89     signatures from both the renter and the host. The connection is then closed.
    90  
    91  File Contract Creation
    92  ----------------------
    93  
    94  A few decisions were made regarding the file contract protocol. The first is
    95  that the renter should not sign the file contract until the host has formally
    96  accepted the file contract. The second is that the host should be the last one
    97  to sign the file contract, as the renter is the party with the strong
    98  reputation system.
    99  
   100  Instead of sending a whole transaction each time, the transaction is sent
   101  piecemeal, and only the new parts at each step are sent to the other party.
   102  This minimizes the surface area of data for a malicious party to manipulate,
   103  which means less verification code, which means less chances of having a bug in
   104  the verification code.
   105  
   106  The renter pays for the siaPrimefund fee on the host's collateral and contract fee.
   107  If a renter opens a file contract and then never uses it, the host does not
   108  lose money. This does put the renter at risk, as they may open up a file
   109  contract and then watch the host leave, but the renter is spreading the risk
   110  over communications with many hosts, and has a reputation system that will help
   111  ensure that the renter is only dealing with upstanding hosts.
   112  
   113  1. The renter makes an RPC to the host, opening a connection. The connection
   114     deadline should be at least 360 seconds.
   115  
   116  2. The host sends the renter the most recent copy of its settings, signed. If
   117     the host is not accepting new file contracts, the connection is closed.
   118  
   119  3. The renter sends a notice of acceptance or rejection. If the renter accepts,
   120     the renter then sends a funded file contract transaction without a
   121     signature, followed by the public key that will be used to create the
   122     renter's portion of the UnlockConditions for the file contract.
   123  
   124  4. The host sends an acceptance or rejection of the file contract. If the host
   125     accepts, the host will add collateral to the file contract, and will send
   126     the renter the inputs + outputs for the collateral, followed by any new
   127     parent transactions. The length of any of these may be zero.
   128  
   129  5. The renter indicates acceptance or rejection of the file contract. If the
   130     renter accepts, the renter will sign the file contract and send the
   131     transaction signatures to the host. The renter will also send a signature
   132     for a no-op file contract revision that follows the file contract.
   133  
   134  6. The host may only reject the file contract in the event that the renter has
   135     sent invalid signatures, so the acceptance step is skipped. The host signs
   136     the file contract and sends the transaction signatures to the renter, and
   137     the host creates and sends a signature for the no-op revision that follows
   138     the file contract. The connection is closed.
   139  
   140  File Contract Revision
   141  ----------------------
   142  
   143  1. The renter makes an RPC to the host, opening a connection. The minimum
   144     deadline for the connection is 600 seconds. The renter then sends a file
   145     contract ID, indicating the file contract that is getting revised during the
   146     RPC.
   147  
   148  2. The host will respond with a 32 byte challenge - a random 32 bytes that the
   149     renter will need to sign.
   150  
   151  3. The renter will sign the challenge with the renter key that protects the
   152     file contract. This is to prove that the renter has access to the file
   153     contract.
   154  
   155  4. The host will verify the challenge signature, then send an acceptance or
   156     rejection. If accepted, the host will send the most recent file contract
   157     revision for the file contract along with the transaction signatures that
   158     validate the revision. The host will lock the file contract, meaning no
   159     other changes can be made to the revision file contract until this
   160     connection has closed.
   161  
   162     A loop begins. The host sends the most recent revision of the host settings
   163     to the renter, signed. The settings are sent after each iteration of the
   164     loop to enable high resolution dynamic pricing for the host, especially for
   165     bandwidth.
   166  
   167  6. The renter may reject or accept the settings + revision. A specific
   168     rejection message will gracefully terminate the loop here. The renter will
   169     send an unsigned file contract revision followed by a batch of modification
   170     actions which the revision pays for. Batching allows the renter to send a
   171     lot of data in a single, one-way connection, improving throughput. The
   172     renter will send a number indicating how many modifications will be made in
   173     a batch, and then sends each modification in order.
   174  
   175     A single modification can either be an insert, a modify, or a delete. An
   176     insert is an index, indicating the index where the data is going to be
   177     inserted. '0' indicates that the data is inserted at the very beginning, '1'
   178     indicates that the data will be inserted between the first and second
   179     existing sectors, etc. The index is followed by the 4MB of data. A modify is
   180     an index indicating which sector is being modified, followed by an offset
   181     indicating which data within the sector is being modified. Finally, some
   182     data is provided indicating what the data in the sector should be replaced
   183     with starting from that offset. The offset + len of the data should not
   184     exceed the sector size of 4MB. A delete is an index indicating the index of
   185     the sector that is being deleted. Each operation within a batch is atomic,
   186     meaning if you are inserting 3 sectors at the front of the file contract,
   187     the indexes of each should be '0', '1', '2'.
   188  
   189  7. The host indicates either acceptance or rejection of the new revision.
   190  
   191  8. The renter signs the revision and sends the signature to the host.
   192  
   193  9. The host signs the revision and sends the signature to the renter. Both
   194     parties submit the new revision to the transaction pool. The connection
   195     deadline is reset to 600 seconds (unless the maximum deadline has been
   196     reached), and the loop restarts.
   197  
   198  File Contract Renewal
   199  ---------------------
   200  
   201  1. The renter makes an RPC to the host, opening a connection. The minimum
   202     deadline for the connection is 600 seconds. The renter then sends a file
   203     contract ID, indicating the file contract that is getting revised during the
   204     RPC.
   205  
   206  2. The host will respond with a 32 byte challenge - a random 32 bytes that the
   207     renter will need to sign.
   208  
   209  3. The renter will sign the challenge with the renter key that protects the
   210     file contract. This is to prove that the renter has access to the file
   211     contract.
   212  
   213  4. The host will verify the challenge signature, then send an acceptance or
   214     rejection. If accepted, the host will send the most recent file contract
   215     revision for the file contract along with the transaction signatures that
   216     validate the revision. The host will lock the file contract, meaning no
   217     other changes can be made to the revision file contract until this
   218     connection has closed. The host sends the most recent revision of the host
   219     settings to the renter, signed. If the host is not accepting new file
   220     contracts, the connection is closed.
   221  
   222  5. The renter either accepts or rejects the settings. If accepted, the renter
   223     sends a funded, unsigned file contract to the host, containing the same
   224     Merkle root as the previous file contract, and also containing a renewed
   225     payout with conditional payments to the host to cover the host storing the
   226     data for the extended duration.
   227  
   228  6. The host will accept or reject the renewed file contract. If accepted, the
   229     host will add collateral (and miner fees if desired) and send the inputs +
   230     outputs for the collateral, along with any new parent transactions. The
   231     length of any of these may be zero.
   232  
   233  7. The renter will accept or reject the host's additions. If accepting, the
   234     renter will send signatures for the transaction to the host. The renter will
   235     also send a signature for a no-op file contract revision that follows the
   236     file contract.
   237  
   238  8. The host may only reject the file contract in the event that the renter has
   239     sent invalid signatures, so the acceptance step is skipped. The host signs
   240     the file contract and sends the transaction signatures to the renter, and
   241     the host creates and sends a signature for the no-op revision that follows
   242     the file contract. The connection is closed.
   243  
   244  Data Request
   245  ------------
   246  
   247  1. The renter makes an RPC to the host, opening a connection. The connection
   248     deadline is at least 600 seconds. The renter will send a file contract id
   249     corresponding to the file contract that will be used to pay for the
   250     download.
   251  
   252  2. The host will respond with a 32 byte challenge - a random 32 bytes that the
   253     renter will need to sign.
   254  
   255  3. The renter will sign the challenge with the public key that protects the
   256     file contract being used to pay for the download. This proves that the
   257     renter has access to the payment.
   258  
   259  4. The host will verify the challenge signature, and then send an acceptance or
   260     rejection. If accepted, the host will send the most recent file contract
   261     revision followed by the signatures that validate the revision. The host
   262     will lock the file contract, preventing other connections from making
   263     changes to the underlying storage obligation.
   264  
   265     A loop begins. The host sends the most recent external settings to the
   266     renter, signed. The settings are sent each iteration to provide high
   267     resolution dynamic bandwidth pricing.
   268  
   269  5. The host will send the renter the most recent file contract revision, along
   270     with the signatures that validate the revision.
   271  
   272     A loop begins, which will allow the renter to download multiple batches of
   273     data from the same connection. The host will send the host settings, and the
   274     most recent file contract revision transaction. If there is no revision yet,
   275     the host will send a blank transaction. The host is expected to always have
   276     the most recent revision (the host signs last), the renter may not have the
   277     most recent revision.
   278  
   279  6. The renter will accept or reject the host's settings. If accepting, the
   280     renter will send a file contract revision, unsigned, to pay for the download
   281     request. The renter will then send the download request itself.
   282  
   283  7. The host will either accept or reject the revision.
   284  
   285  8. The renter will send a signature for the file contract revision.
   286  
   287  9. The host sends a signature for the file contract revision, followed by the
   288     data that was requested by the download request. The loop starts over, and
   289     the connection deadline is reset to a minimum of 600 seconds.